High Efficiency Video Coding (HEVC) is a new coding standard that has been developed in recent years. In the High Efficiency Video Coding (HEVC) system, the fixed-size macroblock of H.264/AVC is replaced by a flexible block, named coding unit (CU). Pixels in the CU share the same coding parameters to improve coding efficiency. A CU may begin with a largest CU (LCU), which is also referred as coded tree unit (CTU) in HEVC. In addition to the concept of coding unit, the concept of prediction unit (PU) is also introduced in HEVC. Once the splitting of CU hierarchical tree is done, each leaf CU is further split into one or more prediction units (PUs) according to prediction type and PU partition. Several coding tools for screen content coding have been developed. These tools related to the present invention are briefly reviewed as follow.
FIG. 1A illustrates an exemplary adaptive Inter/Intra video coding system incorporating in-loop processing according to HEVC. For Inter-prediction, Motion Estimation (ME)/Motion Compensation (MC) 112 is used to provide prediction data based on video data from other picture or pictures. Switch 114 selects Intra Prediction 110 or Inter-prediction data and the selected prediction data is supplied to Adder 116 to form prediction errors, also called residues. The prediction error is then processed by Transformation (T) 118 followed by Quantization (Q) 120. The transformed and quantized residues are then coded by Entropy Encoder 122 to form a video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packed with side information such as motion, mode, and other information associated with the image area. The side information may also be subject to entropy coding to reduce required bandwidth. Accordingly, the data associated with the side information are provided to Entropy Encoder 122 as shown in FIG. 1A. When an Inter-prediction mode is used, a reference picture or pictures have to be reconstructed at the encoder end as well. Consequently, the transformed and quantized residues are processed by Inverse Quantization (IQ) 124 and Inverse Transformation (IT) 126 to recover the residues. The residues are then added back to prediction data 136 at Reconstruction (REC) 128 to reconstruct video data. The reconstructed video data may be stored in Reference Picture Buffer 134 and used for prediction of other frames.
As shown in FIG. 1A, incoming video data undergoes a series of processing in the encoding system. The reconstructed video data from REC 128 may be subject to various impairments due to a series of processing. Accordingly, various in-loop processing is applied to the reconstructed video data to improve visual quality before the reconstructed video data are stored in the Reference Picture Buffer 134 in order to improve video quality. In the High Efficiency Video Coding (HEVC) standard being developed, Deblocking Filter (DF) 130 and Sample Adaptive Offset (SAO) 131 have been developed to enhance picture quality. The in-loop filter information may have to be incorporated in the bitstream so that a decoder can properly recover the required information. Therefore, in-loop filter information from SAO is provided to Entropy Encoder 122 for incorporation into the bitstream. In FIG. 1A, DF 130 is applied to the reconstructed video first and SAO 131 is then applied to DF-processed video. However, the processing order among DF and SAO can be re-arranged.
A corresponding decoder for the encoder of FIG. 1A is shown in FIG. 1B. The video bitstream is decoded by Entropy Decoder 142 to recover the transformed and quantized residues, SAO information and other system information. At the decoder side, only Motion Compensation (MC) 113 is performed instead of ME/MC. The decoding process is similar to the reconstruction loop at the encoder side. The recovered transformed and quantized residues, SAO information and other system information are used to reconstruct the video data. The reconstructed video is further processed by DF 130 and SAO 131 to produce the final enhanced decoded video.
The coding process in HEVC encodes or decodes a picture using a block structure named Largest Coding Unit (LCU). The LCU is adaptively partitioned into coding units (CUs) using quadtree. In each leaf CU, DF is performed for each 8×8 block and in HEVC, the DF is applied to 8×8 block boundaries. For each 8×8 block, horizontal filtering across vertical block boundaries (also called vertical edges) is first applied, and then vertical filtering across horizontal block boundaries (also called horizontal edges) is applied. During processing of a luma block boundary, four pixels on each side of the boundary are involved in filter parameter derivation, and up to three pixels on each side of the boundary may be changed after filtering.
FIG. 2 illustrates the pixels involved in the DF process for a vertical edge 210 between two blocks, where each smallest square represents one pixel. The pixels on the left side (i.e., pixel columns p0 to p3 as indicated by 220) of the edge are from one 8×8 reconstructed block, and the pixels on the right side (i.e., pixel columns q0 to q3 as indicated by 230) of the edge are from another 8×8 reconstructed block. In the DF process according to HEVC, the coding information of the two 8×8 blocks is used to calculate the boundary filter strength (also referred to as BS or boundary strength) of the edge first. After the boundary filter strength is determined, columns p0-p3 and q0-q3 of the reconstructed pixels are used to derive filter parameters including filter on/off decision and strong/weak filter selection.
FIG. 3 illustrates the boundary pixels involved in the DF process for a horizontal edge 310, where each smallest square represents one pixel. The pixels on the upper side (i.e., pixel rows p0 to p3 as indicated by 320) of the edge are from one 8×8 reconstructed block, and the pixels on the lower side (i.e., pixel rows q0 to q3 as indicated by 330) of the edge are from another 8×8 reconstructed block. The DF process for the horizontal edge is similar to the DF process for the vertical edge.
According to HEVC, three levels (i.e., 2, 1 and 0) of boundary filter strength may be used. For a strong boundary (i.e., more visible boundary), a stronger deblocking filter is used to cause smoother boundary. A stronger DF is indicated by BS=2 and a weaker DF is indicated by BS=1. When BS is equal to 0, it indicates no deblocking filter.
Currently, extensions of HEVC (High Efficiency Video Coding) are being developed, including screen content coding (SCC) and 3D extensions. The screen content coding targets at coding screen captured content, with non-4:2:0 color formats, such as 4:2:2 and 4:4:4, and video data with higher bit-depths such as 12, 14 and 16 bit-per-sample while the 3D extension targets at the coding of multi-view video with depth data.
During the Course of SCC development, various video coding tools have been described, including the “Intra picture block copy” (IntraBC) technique. The IntraBC technique was first disclosed in JCTVC-M0350 (Budagavi et al., AHG8. Video coding using Intra motion compensation, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC 1/SC 29/WG11 13th Meeting: Incheon, KR, 18-26 Apr. 2013, Document: JCTVC-M0350). An example according to JCTVC-M0350 is shown in FIG. 4, where a current coding unit (CU, 410) is coded using Intra MC (motion compensation). The prediction block (420) is located from the current CU and a displacement vector (412). In this example, the search area is limited to the current CTU (coding tree unit), the left CTU and the left-left CTU. The prediction block is obtained from the already reconstructed region. Then, the displacement vector, also named motion vector (MV) or block vector (BV), and residual for the current CU are coded. It is well known that the HEVC adopts CTU and CU block structure as basic units for coding video data. Each picture is divided into CTUs and each CTU is reclusively divided into CUs. During prediction phase, each CU may be divided into multiple blocks, named prediction units (PUs) for performing prediction process.
Modifications based on JCTVC-M0350 is disclosed in JCTVC-N0256 (Pang et al., Non-RCE3: Intra Motion Compensation with 2-D Mrs, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC 1/SC 29/WG11 14th Meeting: Vienna, AT, 25 Jul.-2 Aug. 2013, Document: JCTVC-N0256) to allow both horizontal and vertical BV components be non-zero. Furthermore, BV coding methods are disclosed in JCTVC-N0256. One method uses the left or above BV as the BV predictor and codes the resulting BVD (BV differences). A flag is signaled first to indicate whether the BVD is zero. When BVD is not zero, an exponential-Golomb of 3rd order code is used to encode the remaining absolute level of the BVD. The sign of BVD is coded using a flag. According to another method, no predictor is used and the BV is coded using the exponential-Golomb codes used for BVD coding in HEVC.
JCTVC-N0256 also discloses some pipeline friendly approaches. For example, no interpolation filters are used. Furthermore, the BV search area is restricted. In one example, the search area is restricted to the current CTU and the left CTU. In another example, the search area is restricted to the current CTU and the rightmost 4 columns of the left CTU.
In SCM-2.0 (Joshi et al., Screen content coding test model 2 (SCM 2), Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP3 and ISO/IEC JTC 1/SC29/WG11, 18th Meeting: Sapporo, JP, 30 Jun.-9 Jul. 2014, Document: JCTVC-R1014), the block vector (BV) coding is modified to use the neighboring BVs and coded BVs as BV predictor (BVP) according to JCTVC-R0309 (Pang, et al., Non-SCCE1: Combination of JCTVC-R0185 and JCTVC-R0203, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP3 and ISO/IEC JTC 1/SC29/WG11, 18th Meeting: Sapporo, JP, 30 Jun.-9 Jul. 2014, Document: JCTVC-R0309). The BV predictor is derived similar to the AMVP scheme in HEVC. The predictor candidate list is constructed by first accessing in order the spatial neighboring blocks a1 and b1 as shown in FIG. 5. If any of the spatial neighbors does not have a block vector, the last 2 coded BVs, which are initialized with (−2*CU width, 0) and (−CU width, 0), are used to fill into the block vector candidate list to make it contain two different entries. To avoid the need for the line buffer, the above BV outside the current CTU is considered unavailable. The last two coded BVs are reset to (0, 0) for each CTU to prevent the data dependency.
Currently, the BV for IntraBC coded block is at integer accuracy, which is different from MV for Inter coded block, which is at quarter-pel accuracy.
According to SCM-2.0, deblocking filter process will be executed along block boundaries after the reconstruction process similar to the conventional HEVC. A boundary filter strength (BS) will decide whether strong filtering, weak filtering or no filtering should be applied to a boundary. In HEVC, if an edge needs deblocking filter, different block boundary filtering strengths (BS) are assigned to block boundaries with different properties. In summary, the following BS decision process is applied:                Step 1. Test 1 corresponding to whether any block on a side of the block boundary is Intra coded is performed. If the result is asserted (i.e., result=yes), BS is assigned a value equal to 2 and the BS decision process is terminated. Otherwise, the BS decision process goes to step 2.        Step 2. Test 2 corresponding to whether the block boundary is also a transform block edge and at least one block on a side of the block boundary has non-zero coefficients is performed. If the result is asserted (i.e., result=yes), BS is assigned a value equal to 1 and the BS decision process is terminated. Otherwise, the BS decision process goes to step 3.        Step 3. Test 3 on coding parameters related to the two blocks on two sides of the block boundary is performed. BS is assigned a value equal to 1 or 0 based on the test 3 result. Coding parameters such as RefList (reference picture list, also referred to as reference list), RefIdx (reference picture index, also referred to as reference index), the number of MVs used, and MV differences of both sides may be used for test 3. For example, when two blocks refer to the same picture and the difference between two MVs are smaller than one integer pixel (either component), BS is set to 0. Otherwise BS is set to 1.        
For a boundary edge in the case when an IntraBC coded block at one side of the block boundary is involved, the deblocking filter BS decision according to the current HEVC practice treats the IntraBC coded block as an Intra coded block. Accordingly, the BS for the related edges is assigned a value equal to 2 for the deblocking filter operation if any of the two blocks next to the block boundary is IntraBC coded.
According to the current deblocking filter practice involved with IntraBC coded blocks, the filtering process treats the IntraBC coded block in a way similar to Intra block. Therefore, it may not result in desirable visual quality when IntraBC coded blocks are involved. Accordingly, it is desirable to develop deblocking filter process for block boundary associated with IntraBC coded blocks to improve visual quality.